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Abstract 

The nine-month study reported in this paper used mixed methods data and a 
qualitative analysis to examine the skill and perceptual development of 17 
graduate design students. Individual differences, perceptions and preferences 
that apparently promoted rapid and productive development included: design 
efficacy, mastery goals, preference for cognitive challenge and tolerance for 
risk-taking. Novice learners benefited from content and context familiarity, 
but as they developed competence, they gained from choosing less familiar 
tasks and content. Features of the learning environment identified as 
contributing to novice designers ’ knowledge and skill development were: 
authentic projects, detailed assignment specifications, multiple types and 
levels of feedback, and clear alignment with professional performance 
standards. These findings inform the strategic design of instructional 
opportunities for novice designers and similarly complex applied 
professional fields. 
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Introduction 

Instructional design (ID) competence has alternately been presented as an art, craft, or scientific 
process or as some combination of these (e.g., Clark & Estes, 1998; Gibbons, 2003; Merrill, 2007; 
Parrish, 2005; Rowland, 1991; Smith & Boling, 2009). Similar to engineering, architecture and 
other design fields, ID calls on a complex applied skill set that must be used adaptively in ever- 
changing contexts of professional practice (Hardre, 2003). In such a diverse field with myriad 
applications, how can foundational coursework in ID effectively prepare students for professional 
practice? This paper examines the effects of a subset of instructional, perceptual and 
environmental factors on the development of ID competence. The educational implications of 
these factors highlight the need to consider their interdependent relationships, rather than treating 
them as independent of each other. 

ID is “increasingly influential in job markets (Atal, 2007; Hempel, 2006) and in academic circles 
(Woyke, 2007)” (Smith & Boling, 2009, p. 4). The field of ID has been challenged to 
acknowledge a broader set of frameworks that underscore the social, emotional, aesthetic and 
organisational complexity of authentic human interactions (Gainert, 1997; Norman, 2004). Even 
with the increase in exposure to the skill of ID, little research has examined the influence of 
instructional, perceptual and environmental features within design courses. This study investigated 
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the role of student perceptions in the development of professional competence toward expertise in 
a complex, applied design skill. 

Background 

Development of Expert and Novice Practice 

The nature of expertise across applied fields, such as design, has been framed both in abstract 
definition and in terms of the practice of expert professionals. Expertise includes extensive 
knowledge, but also extends to application of that knowledge, in strategic skill that is easier and 
more fluid for experts than for novices (Anderson, 2000; Dreyfus & Dreyfus, 2005). Expertise has 
been theorised on a number of developmental continua, with various stages and sets of labels (e.g., 
Chi, 2006; Cross, 2007; Dreyfus & Dreyfus, 2005). Some definitions of expertise hold up doing 
different and more challenging tasks well as the demonstration of greater expertise than doing 
simpler tasks (Bereiter & Scardamalia, 1993), often including elements of adaptability that lead to 
potential for innovation (Hatano, 1990). Other definitions of expertise posit that doing the same 
tasks more quickly (automatically) demonstrates greater expertise than doing them more slowly 
(deliberately) (Anderson, 2000). One estimate is that an individual requires 10 years and 10,000 
hours of explicit and deliberate practice to become expert in an applied professional skill 
(Ericsson, Krampe & Tesch-Romer, 1993). However, even over time, practice must increase in 
complexity and challenge or it can result in little to no change in true skill (Fadde, 2009). Moving 
learners toward greater expertise, or competence attainment in a complex, applied field, requires 
providing clear performance standards, with continuous practice, and standards-attentive feedback 
on their developing expertise (Fadde, 2009). The schematic organisation which experts develop is 
highly situated, dependent on its strategic context-of-application (Ross, Shafer & Klein, 2006), and 
it includes multiple potential strategies to solve problems (Anderson, 2000; Bereiter & 

Scardamalia, 1993; Bransford, Brown & Cocking, 1999; Goel & Pirolli, 1992). 

In ID specifically, experts have more complex thinking, more systematic approaches to problem 
representation, and more iterative reflection and monitoring than novices (Perez & Emery, 1995). 
Competent designers use data to inform and support analysis, design and evaluation decisions 
(Dick, Carey & Carey, 2009; Piskurich, 2006; Richey, Fields & Foxon, 2001). In contrast, novice 
designers tend to fail at making informed design decisions and focus on producing flashy 
instruction without considering the implications to learning (e.g., use of modalities, distraction) 
(Alessi & Trollip, 2001). Traditionally, ID has been presented as a strategic problem-solving task 
(Richey, Fields & Foxon, 2001; Morrison, Ross & Kemp, 2007). With its emphasis on a 
replicable, systematic process (Brown & Green, 2006; Cennamo & Kalk, 2004), the ID literature is 
relatively silent on the role of the individual designer (Smith & Boling, 2009). However, 
individual differences in philosophy, background, perspective and experience could account for 
variance of interpretations, application, precision and creativity in design (Owen, 2007). With 
increased attention to creativity, aesthetics and fine points of judgment (Molenda & Boling, 2008; 
Rowland, 1995), it becomes more important to investigate the developing practice of today’s 
novices who will become tomorrow’s designers (Rowland, 2008). With this forward-looking 
perspective, an element critical to innovative design competence is the development and nurturing 
of designers’ independent judgment and range of application potential (Cross, 2007; Nelson & 
Stolterman, 2003; Schon, 1987). 


ID Learning and Skill Development 

While the practice of expert designers is given much research attention, far less investigation has 
been done on the development from novice to expert (Dorst & Reyman, 2004). A few studies have 
investigated learning and development of ID knowledge and skill among students of ID (Ertmer & 
Cennamo, 1995; Ge & Hardre, 2010; Hardre, Ge & Thomas, 2007; Quinn, 1994), and of designers 
in specialised contexts (e.g., Christensen & Ogusthorpe, 2004; Rowland, 1991). Some research has 
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also been done on graduate teaching assistants learning ID as a strategic skill for transfer to their 
own professional teaching practice (Hardre & Chen, 2005, 2006). Yet there exists comparatively 
little understanding of skill development, adoption, and transformation of practice (Dorst & 
Reyman, 2004). While it makes sense intuitively that project-based instruction is best for ID skills, 
little research exists that explains how and why that pedagogical approach works. 


Collaboration and Social Learning 

The social environment plays a significant role in learning and development, as learners 
experience peer and personal success and build personal efficacy for new knowledge and skill 
areas (Bandura, 1997). Teaching and explicitly modeling expertise in any field requires the expert 
to make the invisible visible, to make implicit internal thinking processes explicit and apparent to 
learners (Fadde, 2009). Collaborative and team-based learning activities deepen intellectual 
development as individuals challenge one another’s perspectives, through mutual inquiry and 
reflection (Lawrence, 2002). This type of learning environment provides multi-source competence 
feedback that stimulates metacognitive awareness and self-monitoring. Through giving and 
receiving feedback, learners practice the application of cognitive and affective references to the 
professional standards and principles of practice. Dynamic group engagement creates social spaces 
that facilitate inquiry, linking learners and instructors as partners in unique knowledge creation 
(York, 2005). In addition to the role of collaboration in learning design, collaboration is a critical 
skill in professional design practice (Cennamo & Kalk, 2004; Fadde, 2009; Richey, Fields & 
Foxon, 2001), so it is a component of design expertise. 

Project-Based Instruction 

While general learning theory informs characteristics of project-based instruction for practice (e.g., 
authenticity, alignment with target performance standards), numerous questions remain that could 
inform design specifically for learning in design. What elements of project-based instruction and 
activities most effectively support ID skill development, and for what types of learners? What 
constitutes the best balance of various features of instruction in project-based design, such as: 
support and autonomy, constraint and choice, experimentation and scaffolding, and self¬ 
monitoring with feedback and coaching? How does that balance shift for different groups of 
learners, types of design goals, and contexts-of-use? Given the complexity of ID for new learners, 
and the interaction of design and content knowledge, does content familiarity enable or constrain 
the development of novices’ design skills? Are there individual difference characteristics that 
interact with various environmental features in ways that support or thwart the development of 
design competence? Given the various definitions of, and foundations for, design competence, do 
the answers to these previous questions depend on how we judge design quality, or do they 
converge? 

Research Purpose 

This study sought to examine learners’ perceptions and backgrounds, along with key features of 
the design of the learning environment, to identify their individual and interactive contributions to 
students’ competence development in the field of instructional design. 


Methods 

Participants 

Participants were graduate students enrolled in two sequential design courses: ID I (N= 17) and ID 
II (7V=1 1, who continued from ID I). Eleven of the seventeen progressed and maintained the cohort 
from the Fall semester to the Spring semester in the same academic year. Ages ranged from 23 to 
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50 years and the initial group included 10 females (60%) and 7 males (40%). Ethnicity of 
participants included: Anglo/White («=10), African American/Black (n= 1), Latino/Hispanic («=1), 
and Asian/Chinese/Korean («=5). The participants’ majors and status ranged from first-year ID 
master’s students to doctoral students in fine arts and psychology. However, all of the participants 
were characterised (by themselves and the researchers) as novice designers, and all sought to 
become competent in ID, to promote achievement of their career and professional goals. 

Course Context and Structure 

The two sequential courses (ID I & II) are first-year graduate courses in ID. Their scope is process, 
principles and application of ID, including opportunities to experience the systematic process of 
designing and developing replicable instruction. The foundational content for the course was in 
three textbooks (Dick, Carey & Carey, 2009; Morrison, Ross & Kemp, 2007, Smith & Ragan, 
2005), among the most used and recommended texts for university-based ID courses (Reiser, 
Mackal & Sachs, 2005). Activities were grounded in adult learning theory and instructional 
principles (e.g., Gagne & Medsker, 1996; Knowles, Holton & Swanson, 1998; Merriam, Caffarella 
& Baumgartner, 2006). 

The project-based courses provide students with guided questions and incremental design tasks 
(i.e., project prospectus, needs analysis, learner analysis, task analysis, instructional plan and 
evaluation) to assist them in applying principles and developing adaptive competence in 
application and alignment. In addition to individual ID projects, students also collaboratively 
engaged in analyses of professional designs and case studies. These group projects required 
students to collaboratively identify complex problems, share multiple perspectives, and negotiate 
final strategies. 

Productivity demands increased over the two-semester sequence. In Semester One, students 
produced one complete design packet, any content, media and learner group. In Semester Two, 
students produced four complete design packets, with required changes in content, media, context 
and learner group across the four designs. The four design assignments also presented increasing 
complexity in their target instructional outcomes. This assignment structure required both 
increased complexity and increased speed of design performance to demonstrate competence 
development. Students received three types of feedback on each design document: 

(i) peer critique and feedback from two to four classmates; 

(ii) general feedback from the instructor to the class on common errors made on the 
assignments (given out in the class notes and discussed in class); and, 

(iii) individualised feedback from the instructor on the student’s specific design work (1-2 
pages, handwritten, private). 

Both courses required students to provide rationales (expose their design reasoning) to justify their 
strategic design choices. They also required metacognitive reflections, challenging learners to 
thoughtfully self-evaluate (in light of feedback from the instructor and peers), and to track their 
own knowledge, skill and competence development over the 9-month study period. 


Data Collection Procedures 

Students were invited to participate by the second researcher (not the instructor), and all agreed. 
Questionnaire instruments were administered online through the SurveyMonkey' secure digital 
administration system. They were completed during class sessions scheduled a priori in the study 
design, with administration supervised by the second researcher. Students submitted assignments 
for the research analysis privately on digital media. All instruments and activities were approved 
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by the institutional review board and all followed human subject protection protocols. Participant 
confidentiality was protected throughout the study. 

Mixed-method data collection included in-class observations, questionnaires, student assignments, 
and structured interviews. Data was collected across two semesters to track growth and 
development of ID skills and perceptions of competence in ID, in addition to identifying and 
verifying apparent influences on that development. Assignments and data collection activities and 
sources for ID I and II are shown in Table 1. 


Table 1 

Data Collection Activities/Sources 


Course 

Week in 
Study 

Data Collection Event 

ID I 

2 

Knowledge & Perceptions Questionnaire Packet Time 1 


3 

Obs #1: Participation, Interaction 


4 

Design Doc #1 submitted, Critique & feedback 


5 

Obs #2: Participation, Interaction 


7 

Obs #3: Participation, Interaction 

Design Doc #2 submitted, Critique & feedback 


8 

Obs #4: Participation, Interaction 


9 

Design Doc #3 submitted, Critique & feedback 


10 

Obs #5: Participation, Interaction 


11 

Design Doc #4 submitted, Critique & feedback 

SME Report (written & verbal) 


13 

Knowledge & Perceptions Questionnaire Packet Time 2 

Obs #6: Participation, Interaction 


14 

Metacognitive Essay Time 1 

Revised ID I Design Packet 

ID II 

20 

Knowledge & Perceptions Questionnaire Packet Time 1 


23 

Obs #1: Participation, Interaction 


24 

Design Doc Packet #1 submitted, Critique & feedback 


26 

Design Doc Packet #2 submitted, Critique & feedback 


27 

Obs #2: Participation, Interaction 


28 

Design Doc Packet #3 submitted, Critique & feedback 


29 

Obs #3: Participation, Interaction 


30 

Design Doc Packet #4 submitted, Critique & feedback 


31 

Obs #4: Participation, Interaction 


32 

Obs #5: Participation, Interaction 

SME Report (written & verbal) 


33 

Knowledge & Perceptions Questionnaire Packet Time 3 

Obs #6: Participation, Interaction 


34 

Metacognitive Essay Time 2 

Revised ID II Design Packet 


Notes to Table 1: Obs = observation by researcher 
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Data Sources and Tools 

The range of data sources and assessment tools are described below. Approximately 57 data points 
were collected for each participant, per semester (114 for each 2-semester case). 

Pre and post questionnaire packets 

The questionnaire packets include demographics and a set of perception and response instruments. 
Demographics included age, gender, ethnic group, educational background, degree program and 
status, general work experience and background. The set of perception and descriptor scales 
focused on the students’ individual characteristics, perception of learning and instructional design 
application and ability. The eight parts of the questionnaire measured goals, need for structure, 
need for cognition, perceived design competence, design efficacy, ID knowledge and strategies, 
satisfaction with ID knowledge and strategies, and reflection on the environment with attributions 
for development. Instrument responses (except the open-ended qualitative knowledge assessment) 
were on Likert-type 6-point scales (anchored from 1= Not at all to 6=Strongly Agree; or from 
1 =Not at all true to 6= Very true). All questionnaire instruments and rubrics were previously tested 
in this population (ID graduate students) and demonstrated satisfactory internal consistencies (as 
Cronbach’s alphas of .70 or greater), as well as consistency with multiple other measures of the 
same outcomes (demonstrating validity) (Ge & Hardre, 2010; Hardre, Ge & Thomas, 2007). The 
observation protocols were unique to this study, but like all of the other instruments they were 
designed by grounding in the relevant theoretical literatures and reviewed by multiple experts 
before use, providing confidence of their validity for use in this study. 

Goals. The instrument measuring goals was the Approaches to Learning (ATL), which identifies 
learners’ personal goal structures (Greene & Miller, 1996). Sample items by subscales include: 
learning/mastery (i.e., “I do my work in this class because I like to understand what I am 
learning”), performance-approach (i.e. “I do my work in this class because I like to score higher 
than the other students”), and performance-avoidance (i.e. “I don’t do my work in this class so I 
can avoid looking stupid to others”). 

Need for structure. The 12-item Need for Structure (NFS) scale assesses learners’ preference for 
well-structured events versus tolerance for work that is less structured or uncertain (Neuberg & 
Newsom, 1993). Sample item: “I don’t like situations that are uncertain”. 

Need for cognition. The 18-item Need for Cognition (NFC) scale assesses preference for deep 
thinking and intellectual challenges (Cacioppo, Petty & Kao, 1984). Sample item: “I find 
satisfaction in deliberating hard and long for hours.” 

Perceived design competence. The 3-item Perceived Design Competence (PDC) instrument 
assesses self-perceptions of competence in design (Hardre, 2003). It presents a general stem, with 
three quality descriptors (capable, achieving, and competent). 

Design efficacy. The 8-item Design Efficacy Scale (DES) assesses learners’ perceived capability 
of succeeding at design tasks even when faced with challenges (Hardre, 2003). Sample item: “If a 
task analysis shows that the task is especially complex, I believe that I can handle the design 
challenge.” 

Satisfaction with design knowledge and strategies. Three sets of parallel 4-item perceptions 
subscales assess design students’ personal satisfaction with the adequacy of their design 
knowledge and strategies to meet the needs that confront them (IDS; Hardre & Chen, 2005, 2006). 
Sample item: “I am satisfied with my current knowledge of how to design instruction.” 

Assessment of ID knowledge and strategies. A set of open-ended qualitative response 
instruments (6 items), assessed knowledge of basic components of ID process, and instructional 
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design principles (IDK; Hardre & Chen, 2005, 2006). Sample item: “What do you need to consider 
to design a learning environment in which students can learn successfully?” 

Reflection on environment and development. A set of open-ended response items directly 
addressed learners’ perceptions of the effects of the class learning environment on their own 
knowledge and skill development in ID (end of each semester). Sample item: “What was the 
specific role of the various peer activities in your development (e.g., case analysis team activity, 
peer feedback on assignments, team projects)?” 

Observations. Two independent observers recorded observations of students, individually and in 
groups, across a range of behaviors and characteristics. Both sets of observations produced data in 
quantitative and qualitative forms. Three times each semester, the instructor recorded observations 
of students’ in-class design activities. Dates were set a priori based on the class schedule. Three 
times each semester, the external researcher-designer made random visits for systematic 
observations of learners’ design activities. The researchers used an observation protocol that 
ensured consistency across users and instances. 


Student work 

The student work is a collection of assignments completed throughout the two-semester course 
sequence, along with peer and instructor critiques and feedback. The performance standard was a 
design that enabled either accurate delivery by an appropriately skilled instructor, or accurate 
production by a development team. Student work was a collective and developmental indicator of 
learners’ development toward ID competence. It was evaluated using rubrics based on a range of 
ID paradigms (e.g., Kruger & Cross, 2006; Perez & Emery, 1995; Richey, Fields & Foxon, 2001). 

Individual and group design products. These designs were scored by two independent raters, 
using original product scoring rubrics developed from a range of sources of evaluation of design 
quality: 1) the IBSTPI ID Professional Competencies (essential) and aligned with Dreyfus’s first 
five levels of expertise development (novice to expert); and 2) a set of qualitative descriptive 
characteristics from the ID professional literature. The assignments for each semester are listed 
below. 

Semester 1 project. Students completed one complete ID packet, from analysis through design 
and plan of evaluation, submitted as a set of five incremental documents (three analyses, design 
specifications, evaluation plan), and then revised for a final submission. 

Semester 2 projects. Students completed four complete ID projects, each producing a different 
type of primary learning outcome in ascending complexity over the semester: lesson 1 (declarative 
or concept), lesson 2 (principle or procedure), lesson 3 (cognitive strategy or problem solving), and 
lesson 4 (motivation, affect or psychomotor.) 

Metacognitive essays. A reflective, self-evaluative essay was submitted with the revised projects, 
at the end of each semester, encompassing: self-perception of ID knowledge and performance, 
learning and skill development, how development was demonstrated in design project(s); goals 
and strategies for continued expertise development. 

Weekly “Active Reading” responses. These weekly writing assignments were in response to the 
assignment to “engage and apply the content from the readings. Don’t just summarise” (1-2 pages, 
typed). 

Collaborative tasks. Collaboration and team tasks over the semester include peer critique and 
feedback on design documents at drafts and final stages; and analysis of ID readings, case studies, 
and professional projects. 
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Interviews 

Interviews were conducted using a semi-structured interview protocol, covering: personal learning 
experience, instructional design perspective, perspective on reflections, and class learning 
environment. Interviews were used to confirm and elaborate the other data sources, on apparent 
patterns of skill development and experience of the critique-and-feedback process. Sample 
questions: “What are some of the assignments and projects that you think are particularly useful to 
help you develop expertise in instructional design?”; “Please describe your experience with the 
peer review and feedback in this class.” and “How would you describe yourself as an instructional 
designer?” 


Data Organisation and Analysis 

The interviews and handwritten notes were transcribed by the College’s research centre staff. Data 
was organised as participant cases, in case documents (Excel spreadsheets and Word files). Data 
analysis was qualitative in process, progressing through phases of: coding, cross-referencing, 
comparison and triangulation, and synthesis into patterns and themes (Saldana, 2009; Yin, 2003). 
The two researchers independently scored the design products, ran descriptive statistical tests as 
appropriate on quantitative indicators, and coded the qualitative information, then joined together 
for synthesis. A cause or influence assertion depended on multiple sources of data (e.g., 
quantitative and qualitative, self-report and observer) that converged to support more than a simple 
correlative relationship among factors (Yin, 2003). 

Coding. The two researchers initially coded the data independently from the case files, and also 
listened to the original audio recordings to include nuances of emphasis and language. Then the 
two researchers met to compare their notes and coding and collaboratively merge them into 
preliminary case files. They iteratively returned to the raw data as needed. 

Cross-referencing and triangulation. The triangulation of data was conducted across three 
dimensions: (a) case-level data triangulation, (b) change-trajectory cross-referencing, and (c) case- 
comparison, multi-investigator triangulation. The researchers analysed all the data in parallel for 
each case to seek the trajectory of changes across the two-semester study. At the case level 
dimension, each individual participant’s case data profile was examined by comparing different 
data sources collected within a given time frame. For example, the first semester’s design 
documents and metacognitive essays were compared with the first semester’s interviews, peer 
feedback, and observation notes. At the change trajectory dimension, each individual case was 
examined by type of data and sources, compared over time through the nine-month study period. 
For instance, individuals’ interviews conducted at different points of time were compared, and the 
first semester’s design packet was compared with the second semester’s design packets. At the 
multi-case, multi-rater dimension, different types of data of an individual case were also cross- 
referenced and compared with other cases. Collaborative notes and results of the researchers’ 
observations, interpretations, and reflections, resulting from the multiple dimensions of analysis, 
were distilled to address the research questions. Results are organised by themes and patterns 
relevant to the research purpose. 

Results 

In seeking to uncover individual difference characteristics that appear to influence students’ skill 
and competence development, we found that several types of individual differences appeared to be 
related to (and may have contributed to) design students’ skill and competence development. Some 
of these were matters of explicit choice, some developed over time, and others were more stable 
differences of perception or personality. These influential differences included: content and 
context familiarity (e.g., choosing less familiar content areas), choice of range in design tasks and 
content areas (e.g., choosing more complex and difficult design tasks), and a cluster of quantifiable 
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characteristics of questionnaires (design self-efficacy, mastery goals, preference for cognitive 
challenge, and tolerance for uncertainty and risk-taking). 

Early, their content area familiarity and context-related background helped to scaffold them 
through the design process, as depending on content familiarity in early designs freed up their 
attention and energy for learning the ID content and principles. However, later in their 
development, those who continued to design in content and contexts of familiarity demonstrated 
less rapid ID skill development, not experiencing the same range of varied practice experiences 
and challenge as those who chose content and contexts that were unfamiliar. General background 
and skills also supported the development of some specialised ID competencies over others. For 
example, students with math, science and logic background found analysis skills easier to develop, 
while those with K-12 teaching background tended to skip analysis and move straight to designing 
instructional strategy (and had to build backward to develop analytical reasoning). Those with IT 
background gravitated toward design of media and materials, but found analysis, assessment, and 
rationales more difficult. The challenge in developing well-rounded ID competence was to stretch 
into those ID skills and competencies least supported by background. They did this by taking 
advantage of resources in several ways: (i) by learning from peers’ advice and examples in their 
areas of strength; (ii) by “digging into” the texts and resources for principles and strategies (they 
read most deeply the text sections on their weakest areas); (iii) by going to the instructor for 
additional explanation, advice and resources to strengthen their skills; or (iv) by seeking out expert 
resources and examples (such as from the online resource list provided for the course, or from their 
own searches). 

Another important difference was challenge-seeking, or the degree to which learners chose to 
engage in original design (from analysis up) or some partial design task (such as translating 
existing instruction from face-to-face to e-learning, or updating existing content). The full range of 
design tasks and translations were allowed and supported, as equally authentic tasks. However, 
they differed in the degree to which learners had to engage in original work (vs. depending on 
work done by others). Like the difference in familiar and unfamiliar content areas, the range of 
design tasks that a learner stretched to take on influenced development over the two semesters, 
through diversity and challenge in practice opportunities (also consistent with Bereiter & 
Scardamalia, 1993). Some students had higher efficacy for challenge, more willingness to take 
risks, and more receptivity to experiential learning (expressed in their reflective and strategic 
writing over the study period), and these characteristics seemed to support those who reached 
higher and farther. Those who took safer, smaller tasks, with less original analysis and design 
work required of them did not develop as rapidly as their peers who tried more original, complex 
and difficult design tasks. 

Self-efficacy for the work of design was linked to knowledge and skill development. This 
characteristic developed markedly over time, consistent with spaced practice and incremental 
success on increasingly complex design tasks. Self-efficacy was also related to learners’ desire to 
engage deeply in the cognitive challenge of learning new skills, and their willingness to take risks 
by trying to design in unfamiliar content and context areas, rather than keeping close to the 
familiar. Initial design self-efficacy was related to learners’ prior experience in, or familiarity with, 
ID, but over time it developed rapidly in response to current knowledge and experience. 

Goal orientations stood out as an important individual difference that influenced design 
development. Those who came with performance goals tended to resist the developmental focus 
initially, and simply wanted “a template” a quick, easy formulaic approach into which to insert 
content, not the requisite of developing adaptive skills. Their early assignments tended to be very 
formulaic, and they expected high grades for filling in the blanks, treating the specifications as 
checklists. Their work tended to be technically sound, but cursory rather than thoughtful, well- 
analyzed and well-developed. Those with mastery goals were more likely to ask questions about 
how to apply the principles and address the specifications. Their early assignments tended to be 
less perfect in addressing the specifications, contained more departures from the norm, more 
development of key ideas (even resulting in longer documents as they “chased” ideas). 
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Learners with higher preference for cognitive challenge, tolerance for error and uncertainty, and 
willingness to take risks tended to develop a wider range of design skills and strategies more 
quickly, and finished the course(s) with relatively more skill development than their more 
conservative, risk-averse peers. As indicated above, this cluster of characteristics was also related 
to both mastery goals and design self-efficacy. The instructor promoted mastery goals, tolerance 
for error, uncertainty and risk-taking, throughout the two-semester course sequence, through: 
modeling mastery development, sharing stories of learning from errors, encouraging risk-taking in 
assignments, active coaching, and individualised feedback. 

Design self-regulation (as task management, self-evaluation and monitoring), increased with 
knowledge and skill, and was reflected in both peer critiques and responses to critiques (notably in 
clarity of points and rationales). Individual differences noted above also supported the 
development of self-regulation. Goals and other productive preferences were more stable with 
mature, self-directed (self-aware and self-regulated) learners. Notably early on (Weeks 4-8) there 
was a shift toward mastery goals, and then another forward shift later (Weeks 26-28). 

Limitations 

With six of the 17 students not continuing into ID II, wc have only limited data for them. The 
study would be strengthened by having all 17 participants for the full study period. 


Conclusion 

Our findings are complex, consistent with the nature of ID, the dynamic of active learning in an 
applied classroom with multi-source inputs and authentic project-based tasks. They might be 
summarised by considering the way that assets to development and evidence of that development 
emerged. 

First, overall, the role of individual differences interacting with the learning environment was 
profound, and apparently resulted in measurable, observable change in: ID knowledge 
development, extension and refinement of ID process skills, increased range of application, 
improved transfer flexibility, more adaptive use of systematic design (vs. content knowledge or 
intuition), and selective and strategic transfer. Changes in beliefs and identity were also profound, 
and mapped back to many of those same characteristics that apparently promoted skill 
development. 

Among individual differences and preferences, those who made the choice to stretch outside of the 
comfortable and familiar, beyond the range mandated by the instructor, gained more than their 
peers who chose not to stretch. This stretching produced development across a wider range of ID 
component skills and application to more content areas. Those who developed more dramatically 
were equipped with a cluster of characteristics including: greater self-efficacy, preference for 
cognitive challenge, tolerance for risk and uncertainty, receptivity to experiential learning, mastery 
goals, and receptivity to critique and feedback. These characteristics promoted a host of related 
benefits that supported development, including choosing more and varied tasks, inquiry leading to 
understanding, less perfect work initially but more productive exploration, greater creativity and 
unconventional design, deeper metacognition and productive self-regulation. 


Contribution to the Field of ID 

This study contributes to knowledge in the field in a number of ways. It examined how individual 
differences and perceptions interact to drive the development of expertise differentially for 
learners (per Bereiter & Scardamalia, 1993; Goel & Pirolli, 1992). It included multi-source and 
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mixed methods data for both process indicators and product outcomes, taking into account 
individual learner differences, such as need for cognition (Cacioppo, Petty & Kao, 1984), need for 
structure (Newburg & Newson, 1993) and self-efficacy (Bandura, 1997). It extended the study of 
development in ID competencies over a longer period than most previous studies, and included its 
situatedness in the design of learning environments (Anderson, Reder & Simon, 1996; Bransford, 
Brown & Cocking, 1999; Brown, Collins & Duguid, 1989; Chi, 2006; Dai & Sternberg, 2004). 

Unlike some previous research (e.g., Tabernero & Wood, 1999), this study found task-specific 
self-efficacy strongly related to self-regulation (as self-evaluation and monitoring). This may be in 
part due to the participant group, which unlike many previous studies, were not teachers but 
professional design students. Building on previous research (e.g., Hardre, Ge & Thomas, 2007), it 
added more detailed assessments of perceptions, and revealed a cluster of perceptions that 
apparently position learners for a greater success in learning ID. Beyond confirming previous 
findings (generally from single-semester studies; e.g., Hardre, Ge & Thomas, 2007), this design 
extends the timeline of development to two semesters, enabling systematic assessment and 
analysis over a longer developmental trajectory. 

Beyond the findings of other two-semester studies (e.g., Ge & Hardre, 2010), the present study 
introduced more robust assessments and a more coherent and diverse sample for studying the 
nature and role of community. Like previous studies with professional design students (e.g., 

Ertmer & Cennamo, 1995; Quinn, 1994), the present study notably features peer interactions, and 
the role of the instructor as expert model and coach. However, this study added tools drawing from 
multiple paradigms of ID not addressed in those earlier studies, seen through richer data sources 
(e.g., observations, interviews). Examples here include the nature and role of deliberate practice 
(Ericsson, Krampe & Tesch-Romer, 1993; Fadde, 2009), the role of problem/task complexity 
(Ericsson, Rikers & Paas, 2005), the role of metacognitive processing (Flavell, 1991), and the 
linkages between educational activities and ID practice articulated in the learning environments 
(Quinn, 1992; Rose, 2002). The complex data enabled us to unlock some of the “how” and “why” 
of design features that contributed to the development of both demonstrated and perceived ID 
competence. 


Implications for Research and Design Education 

This study addresses some characteristics of the learning environment for ID and how they align 
with demands of the professional field. These findings inform and invite additional research, to 
further investigate how these characteristics individually and interactively contribute to 
competence development. This study moves toward defining some characteristics of learners and 
learning environments that promote efficient and effective development of ID knowledge and 
skills. One example of such characteristics is the individual difference profile that positions 
novices for success in ID skill development, related to the nature of the field, its content and 
practice. This profile is linked to previous research findings (both in and beyond ID) and addresses 
links between learner and environmental characteristics (such as the cluster of individual and 
perceptual characteristics with features of learning environment design), and between learner 
background and the field of ID. 

Related to the issue of autonomy in applying and integrating feedback into development of design, 
a question that arises is how the novice designers integrate information across phases and types of 
input, demonstrating iterative progress. Theoretically, each design should begin on a higher skill 
level, but the apparent trajectory of these novice designers was not as linear as expected. Over the 
study period, questions emerged on how pivotal characteristics of learners mapped onto evidence 
of competence across key process and product indicators. Even closer tracking of participant 
activity and systematic assessment of multiple versions of design products can provide clearer 
evidence of competence development to illuminate these issues. 
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The field of ID needs more systematic, context-sensitive research on competence development. 
This future research needs to draw on related work in educational psychology, to consider 
instructional components and assessment tools largely neglected to date. Such a body of 
progressive research will support improved design of both traditional and digital learning 
environments intended to develop ID skills and practice. 

Beyond ID alone, these findings bear implications for other similarly complex applied design 
fields, such as engineering and architecture. They illuminate how learners’ individual differences 
interact with the nature of the disciplines and tasks, as highlighted in the learning environment, to 
produce insights on how and why certain students develop more readily or rapidly than others. 
Colleagues may wish to test these findings in related disciplines and similar fields of professional 
practice. 
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